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ABSTRACT 

Five pre~ and postevaluat ions were administered to 
413 therapy aides who had participated in a workplace literacy 
program (which 273 had completed) to upgrade their job-specific 
reading, writing, and oral communication skills in five New York City 
psychiatric hospitals. Four of the measures showed significant 
pre-post gain. The data were then examined to determine whether the 
amount of gain could be predicted by four different kinds of 
variables: (l) student demographics, (2) newspaper-reading practices 
and knowledge of current affairs, (3) number of hours of instruction, 
and (4) students* learning goals. The analysis showed that although 
there was significant pre-post gain on four of the five outcome 
measures, the amount of absolute gain was small. The small amount of 
absolute gain in combination with the relative heterogeneity of the 
population led to difficulty in obtaining significant predictors of 
gain. However, subsequent informal analysis of the data revealed that 
a greater amount of gain was associated with the following factors: a 
tendency to follow current affairs, frequency of newspaper reading, a 
match between the worker's learning goal and the purpose of the 
course, higher educational level and educat ional credentials , and 
possibly lower age and fewer years in the job. Number of hours in 
instruction predicted gain for only two of the outcome measures, only 
one of which measured actual literacy skills. The research suggested 
that when the sample is heterogeneous, as it typically is in the 
adult literacy population, small group comparisons are more promising 
than large group analyses (KC) 
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Relationships between Student Variables and Pre-Post Gain 
in a Workplace Literacy Program 1 

Direct care workers in five state-operated psychiatric 
hospitals in New York City attended a 96-hour workplace literacy 
course at the work site to prepare for new job tasks. The 
participants, whose job title was Mental Hygiene Therapy Aide 
(MHTA) , were expanding their job functions as part of an agency- 
wide shift in treatment approach. The new mission, called 
psychiatric rehabilitation, requires that professional and 
paraprofessional staff work together in teams to plan and 
implement the rehabilitation of severely mentally ill patients. 
As a result of this change, therapy aides are expected to use 
literacy skills more than before, for example in reading and 
writing patient progress reports. In the study reported here, 
many of the therapy aides were academically underprepared for 
these functions although in fact most had completed secondary 
education and 33% had earned some college credit. 

Workplace literacy instruction was provided twice a week 
over a six-month period in either 2- or 3-hour classes. The 
teachers used a custom-tailored workplace literacy curriculum 
which focused on job-specific reading, writing and oral 
communication skills. Instruction addressed areas such as job- 
related technical vocabulary and diagnostic terms, oral 

1 The work reported here was funded by the U.S. Department of 
Education under Grant Award #V198A20004. The project was conducted by the 
CUNY Graduate School, the Civil Service Employees Association, the NYS Office 
of Mental Health, and the NYS Governor's Office of Employee Relations. 
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communication about patients' behavior, report-writing skills, 
and reading comprehension of treatment plans and other documents. 
The teachers spent the largest proportion of instructional time, 
on writing skills, the area of greatest need. The employer 
provided full release time for attendance, which was voluntary. 
Further information regarding the program is available in Perin 
(1994) . 

Five measures were administered pre and post. Four of the 
measures showed significant pre-post gain. The purpose of this 
study was to examine whether the amount of gain could be 
predicted by four different kinds of variables: (1) student 
demographics, (2) newspaper-reading practices and knowledge of 
current affairs, (3) number of hours of instruction, and (4) 
students' learning goals, a measure that represented the number 
of different goals held by students upon entry to the program. 

Method 

Participants . A total of 465 therapy aides participated over 
three six-month cycles. The data analyzed in this study came from 
the first two cycles, in which 413 employees enrolled and which 
273 (66%) completed. Seventy-five percent were women, 72% were 
aged 31-50, and almost all were of ethnic minority status 
(African-American or Hispanic) . Eighty-eight percent spoke 
English as a primary language in childhood, and 96% currently 
spoke English as a primary language. Additional student 
background information is provided in the results section below. 
Materials . Of the five pre-post measures, three were job- 
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specific, developed expressly for the program. The MHTA Reading 
Test consisted of a passage of approximately 500 words that 
simulated a patient treatment report, followed by 14 literal and 
inferential short-answer comprehension questions. Alternate 
forms were used so that approximately half of the participants 
received Form A as a pretest and Form B as a posttest, with the 
order reversed for the remaining participants. Different 
versions of the test were used in the first two instructional 
cycles. Only data from the first cycle are reported here. 

The MHTA Writing Test required that participants write a 
description, intended for co-workers, of "the most difficult 
patient or the most interesting patient you've worked with." The 
writing samples were scored holistically on five dimensions: 
content, organization, vocabulary, language use, and mechanics. 

The MHTA Self-Efficacy Scale asked the participants to rate 
their confidence in their ability to accomplish fifteen different 
job-specific literacy tasks. 

The other two measures were the Test of Applied Literacy 
Skills (TALS), Prose section (Kirsch, Jungeblut, & Campbell, 
1991) and the Tennessee Self -Concept Scale (TSCS) (Roid & Fitts, 
1991) . 

Data on student demographics, newspaper-reading practices 
and knowledge of current affairs, and students' learning goals 
were drawn from registration forms. Student demographic 
variables were: (1) number of years in current job; (2) highest 
grade completed by age 21, (3) educational credentials (no 



credential; high school diploma; General Education Development 
(6ED) diploma; and either high school or 6ED diploma plus some 
college credits; (4) age (21-30 years, 31-40, 41-50, or 51-60); 
and (5) sole family provider (yes or no) . 

Two questions regarding frequency of newspaper-reading and 
knowledge of current affairs were taken from the National Adult 
Literacy Study (Kirsch, Jungeblut, Jenkins, & Kolstad, 1993), 
Students were asked how frequently they read a newspaper in the 
English language. They checked one of five choices: every day, a 
few times a week, once a week, less than once a week, and never. 
They were asked to select one of four choices regarding whether 
they followed what is going on in government and public affairs: 
most of the time, some of the time, only now and then, and hardly 
at all. 

Learning goal data were drawn from a question on the 
registration form that asked participants what they hoped to 
achieve as a result of the course. Students were asked to select 
all goais that applied from a choice of seven: to prepare for 
psychiatric rehabilitation functions; to improve in job functions 
in general; to improve in general academic skills, to prepare for 
career advancement; to prepare to enter or re-enter college; to 
improve academic skills in order to help children with homework; 
and "other." For each participant, the variable "goal" was 
computed by adding the number of items selected. 

Number of hours in instruction were taken from the teachers' 
attendance rosters. 
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Procedure . The pretests were administered in the third week of 
the six-month cycle, and the posttests were administered in the 
week prior to the end of the cycle. Participants were informed 
of both pre and post scores in a confidential letter at the end 
of the cycle. No one else at the workplace received this 
information or, indeed any other information regarding individual 
participants' academic standing. All reporting was based on 
group data. 

The pre and post measures were administered by the project 
educational coordinator, using standardized procedures. The 
order of test administration, verbal directions, and time allowed 
per test were constant. 

Registration forms were completed during the first 
instructional session. Attendance data were recorded each 
session by the teachers, under the supervision of the educational 
coordinator. 

Scoring . A research assistant scored all the tests except the 
MHTA Writing Test, which was scored by the educational 
coordinator. Answer keys were used for the MHTA Reading Test, 
and written criteria for holistic scoring were applied to the 
MHTA Writing Test. The maximum score on each of these two tests 
was 20 points. 

Scores on the MHTA Self-Efficacy Test were obtained by 
adding the numbers representing the students' ratings on a 7- 
point scale for each of 15 questions, for a maximum total score 
of 105. The TALS Prose test yielded a maximum scaled score of 
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450. Ninety questions of the Tennessee Self -Concept Scale (TSCS) 
were used. Each answer consisted of a rating on a 5-point scale 
for a total maximum score of 450. 

Data Analysis . The discrete variables (age, newspaper-reading 
practices, and knowledge of current affairs) were examined 
separately in relation to each of the five post scores, with the 
pre score as a covariate. The same procedure was utilized for 
the nominal variables educational credential and sole family 
provider. The continuous variables (number of years in the job, 
highest grade completed by age 21, goal, and number of hours in 
instruction) were each examined using partial correlations with 
the post scores, controlling for the pre score. 

Most of the participants answered the question regarding 
frequency of newspaper-reading by selecting every day or a few 
times a week. For the purpose of this analysis, two categories 
of response were created, every day, and a few times a week or 
less. Similarly, most the respondents selected one of two 
answers for the question regarding whether they followed current 
affairs: most of the time or some of the time. For the purpose 
of the analysis, two categories were created: most of the time, 
and some of the time or less. 

Data are missing for each of the variables studied. In some 
cases, a participant did not take both pre and post tests, and in 
other cases, certain questions on the registration forms were not 
answered. In other cases, attendance data are missing. 
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Results 

Pre-Post Gains , Scores are shown in Table 1 for all students 
who took both pre and post tests. 



Table 1 . Pre and Post Scores 



Test 


N 


Pre Mean 


SB. 


Post Mean 


SD 


t 


MK ?A Reading 


168 


15.5 


3.9 


17.1 


3.0 


4.3* 


MKTA Writing 


181 


11.1 


3.6 


13.5 


3.3 


13.6* 


MHTA Self -Efficacy 


188 


76.7 


12.5 


83.4 


29.1 


3.2* 


TALS Prose 


181 


293.3 


37.9 


290.9 


37.2 


-1.1 


TSCS 


189 


344.4 


34.3 


353.3 


33.8 


3.5* 



* significant pre-post difference, p<.05 

Demographic Variables 

Frequencies for the demographic variables used in this study 
are presented in Table 2 below. 



Table 2. Demographic Variables 

Variable fi 
Number of years in current job 305 

1 or below 17 

2-5 72 

6-10 81 

More than 10 135 

Highest grade completed 339 

8 or below 11 

9-12 307 

Abovo 12 21 

Educational credentials 344 

No H.S. dipl. or GED 36 

H.S. diploma 112 

GED 60 
H.S. or GED, and 

some college credits 129 

College degree 7 

Age 369 

21-30 46 

31-40 129 

41-50 135 

51-60 59 

Sole family provider 371 

Yes 268 

No 103 



Percentage 

5.6 
23.6 
26.6 
44.3 

3.2 
90.6 
6.2 

10.3 
32.6 
17.4 

37.5 
2.0 

12.5 
35.0 
36.6 
16.0 

72.2 
27.8 
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Newspaper-Reading and Knowledge of Current Affairs 

Frequencies for newspaper-reading and knowledge of current 
affairs are shown in Table 3 below. 



Table 3 . Frequency of Newspaper-Reading and Knowledge of Current 
Affairs 



Variable H 

Frequency of newspaper-reading 380 

Every day 222 

A few times a week or less 158 

Follow current affairs 374 
Most of the time , 179 

Some of the time or less 195 



Percentage 

58 
42 

48 
52 



Number of Hours of Instruction 

The number of hours of instruction is shown in Table 4 
below. 



Table 4 . Hours in Instruction 
Number of Hours 

2-40 
41-68 
69-80 
81-96 



M 
330 
83 
81 
85 
81 



Percentage 

25.2 
24.5 
25.8 
24.5 
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Relation Between Amount of Gain, and Other Variables 

Three significant relationships were found between the 
amount of literacy gain and the other variables. The number of 
hours spent in instruction was related to gain on the MHTA 
Reading Test (r = .16, df - 160, p<.02) and the MHTA Self- 
Efficacy Scale (r= .24, df = 166, p<.001). Self-reported 
knowledge of current affairs was significantly related to the 
MHTA Self-Efficacy Scale (F = 4.63, df = 168, 1, 1, p<.03). 

Discussion 

Although there was significant pre-post gain on four of the 
five outcome measures, the amount of absolute gain was small. 
The most robust pre-post difference was found for the MHTA 
Writing Test, which only yielded a mean two and one-half point 
gain from pre to post on a 20 point scale. The small amount of 
absolute gain in combination with the relative heterogeneity of 
the population may explain the difficulty in obtaining 
significant predictors of gain. 

It is possible that relationships between gain and student 
variables were obscured when data for the whole sample were 
analyzed together. Subsequent informal examination of the data 
studied the relation of the student variables to writing scores, 
since writing was the area that received most instructional time 
and produced the strongest gain from pre to post. In order to 
explore further the possibility that the amount of literacy gain 
could be predicted by background variables, two extreme groups 
were created, students who showed pre-post gain on the writing 
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test of 5 or more points (Group A, N=14), and students who showed 
no change or a 1 -point change, up or down, from pre to post 
(Group B, N=18) . All students in both groups had scores of 8 to 
11 on the writing pre test. 

Some interesting trends were identified. The groups were 
alike in some ways and different in others. The two groups 
attended class for the same number of hours (Group A mean 78.36, 
SD 14.74; Group B mean 78.53, SD 9.34). Group A selected a mean 
of 3.79 learning goals (SD 1.74), which was similar to the mean 
of 4.11 (SD 1.82) for Group B. Notably, 86% of Group A included 
the highly specif ic workplace goal of improvement in psychiatric 
rehabilitation, the major purpose of the course, compared to only 
44% for Group B. 

Eighty-six percent of Group A were sole family providers, 
not too different from the 72% in Group B. Both groups hart spent 
many years in the current job title, although Group B showed a 
higher mean (Group A mean 12.36, SD 7.46, Group B mean 16.31, SD 
8.51). Forty-six percent of the employees in Group B had spent 
20 or more years in the same job title, compared with 27% for 
Group A. Related to this finding, Group B contained a greater 
proportion of older workers, as shown in Table 5 below. 
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Table 5. Writing Scores and Age 

Group A (N=14) 

Age Frequency Percentage 

21-30 0 0 

31-40 5 36 

41-50 6 43 

51-60 3 21 



Group B (N=18) 
Frequency Percentage 

0 0 
4 22 

13 72 

1 6 



Seventy-eight percent of Group A compared with 93% of Group 
B held either a high school diploma or a GED. An additional 22% 
of Group A compared with 7% of Group B held a secondary education 
diploma and had also earned some college credits. Thus, Group A 
was a slightly better educated group. This finding is echoed by 
the pattern for highest grade completed. Eighteen percent of 
Group B had completed only 9th or 10th grade, with the remainder 
having completed llth or 12th grade, while all of Group A had 
completed llth or 12th grade. 

The groups differed in their frequency of reading the 
newspaper, and their knowledge of current affairs. Seventy-two 
percent of Group A compared with 56% of Group B indicated that 
they read the newspaper every day. Further, it was particularly 
striking that seventy-nine percent of Group A compared with only 
22% of Group B indicated that they followed government and public 
affairs most of the time. As stated earlier, a significant 
relationship was also found between following current affairs and 
gain on the MHTA Self-Ef f icacy Scale. It is possible that adult 
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students' level of interest in current affairs is an index or 
marker variable for level of success in literacy instruction. 

In summary, while the large group analysis yielded few 
significant relationships, the exploratory comparison of 
"extreme" groups revealed that a greater amount of gain from pre 
to post was associated with a tendency to follow current affairs, 
frequency of newspaper-reading, a match between the worker's 
learning goal and the purpose of the course, higher educational 
level and educational credentials, and, possibly lower age and 
fewer years in the job title. 

It is interesting that number of hours in instruction 
predicted gain on only two of the outcome measures, and only one 
of these, the MHTA Reading Test, measured actual literacy skills. 
Number of hours in instruction did not predict gain in writing 
skills, the area receiving the most attention in the classroom. 
Further research is warranted to discover the reasons for this 
pattern, since adult literacy program planners assume that 
participants will gain from more instruction; retention and drop 
out prevention are major issues in the adult literacy field. 

From a methodological point of view, the findings of the 
current study suggest that when the sample is heterogeneous, as 
it typically is in the adult literacy population, small group 
comparisons are more promising than large group analyses. 
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